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ABSTRACT 

The  Mission  Success  Prediction  Capability  (MSPC)  is  a  Model-Based  Systems  Engineering 
(MBSE)  approach  to  mission  planning,  used  for  the  analysis  of  complex  systems  of  precision 
strike  (air-to-surface)  weapons.  This  report  focuses  on  developing  a  statistical  method  for 
evaluating  the  probability  of  mission  success  in  these  situations.  Bayesian  networks  are  used 
as  a  framework  for  the  calculation  of  the  probability  in  question,  and  a  general  methodology 
for  the  evaluation  is  formed  and  demonstrated  in  this  report. 
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A  Framework  for  the  Statistical  Analysis  of 
Probability  of  Mission  Success  Based 
on  Bayesian  Theory 


Executive  Summary 


The  Mission  Success  Prediction  Capability  (MSPC)  is  a  Model-Based  Systems 
Engineering  (MBSE)  approach  to  mission  planning,  used  for  the  analysis  of  complex 
systems  of  precision  strike  (air-to-surface)  weapons.  This  report  focuses  on  developing 
a  statistical  method  for  evaluating  the  probability  of  mission  success  in  these  situations, 
from  the  type  of  model  used  in  evaluation  to  the  method  for  the  implementation  of  this 
model. 

Research  has  been  conducted  to  assess  which  method  of  calculation  of  probability  is 
best  suited  for  this  capability.  By  comparing  four  valid  models  with  respect  to  the 
project  and  its  needs,  Bayesian  networks  have  been  chosen  as  the  most  appropriate 
statistical  method  for  this  task.  This  is  due  to  many  reasons,  including  the  fact  that 
Bayesian  inference  is  far  more  capable  of  accounting  for  non-holistic  and  sometimes 
subjective  data  than  other  methods,  such  as  logistic  regression. 

While  it  is  sometimes  possible,  and  may  be  necessary  in  future,  to  statistically  assess 
the  dependencies  between  the  events  in  a  system,  expert  opinion  was  used  in  this 
report  to  generate  an  appropriate  demonstrative  network  for  the  probability  of  mission 
success. 

A  general  formula  was  derived  for  the  calculation  of  this  probability  based  on  the 
Bayesian  network  constructed.  Erom  this,  a  Matlab®  function  has  been  developed  to 
demonstrate  how  this  formula  could  be  implemented  in  regard  to  the  system 
constructed  earlier.  Ealse  probability  distributions  are  used  in  this  function  for 
demonstration  purposes.  The  Matlab®  code  gives  a  general  method  for  transferring  the 
network  to  a  program  that  can  calculate  the  probability  of  mission  success.  Software 
specifically  designed  for  Bayesian  networks,  AgenaRisk®,  has  also  been  used  to 
demonstrate  this  formula.  A  major  issue  with  any  probabilistic  model  is  the 
computational  burden  and  methods  for  avoiding  and  ameliorating  this  problem  are 
discussed  in  this  report. 

This  report  covers  the  initial  consideration  of  the  statistical  aspects  of  this  project.  Erom 
here,  data  needs  to  be  collated  and  probability  distributions  generated  to  apply  this 
network  in  practice.  A  method  for  measuring  the  precision  of  the  result  must  also  be 
produced  to  give  credibility  to  the  calculated  mission  success  probability. 
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1.  Introduction 


The  Mission  Success  Prediction  Capability  (MSPC)  is  a  Model-Based  Systems  Engineering 
(MBSE)  approach  to  mission  planning.  It  focuses  on  holistically  analysing  complex 
systems,  more  specifically  those  of  precision  strike  (air-to-surface)  weapons.  The  aim  of  the 
MSPC  is  to  [1]: 

"Use  all  available  data  to  predict  the  success  of  a  given  mission  enabling  Commanders  the  flexibility 
to  make  more  efficient  and  effective  use  of  complex  weapon  systems." 

The  output  of  this  capability  is  intended  to  be  a  software  system  which  combines  all  the 
available  data  to  simulate  the  proposed  mission. 

This  report  covers  the  initial  consideration  of  the  statistical  aspects  of  the  capability,  in 
which  a  framework  for  measuring  the  probability  of  mission  success  of  a  weapon  is 
developed.  This  includes  discussion  on  the  type  of  analysis  method  best  implemented  for 
the  MSPC.  The  merits  of  the  preferred  method  applied  to  analyse  the  probability  of 
mission  success,  a  Bayesian  network,  are  discussed  in  the  literature  review  in  Section  2, 
specifically  in  comparison  to  three  other  methods.  The  other  methods,  which  are  both 
probabilistic  and  non-probabilistic,  are  briefly  explained  and  their  qualities  relevant  to  the 
project  are  discussed. 

The  methodology  of  the  report  covers  the  statistical  development  of  the  Bayesian  network. 
This  includes  the  construction  of  the  physical  structure  of  the  network,  as  well  as  the 
method  with  which  the  probabilities  are  calculated  using  Bayesian  probabilities.  The 
implementation  of  this  methodology  is  then  demonstrated,  with  regards  to  data 
requirements  and  potential  issues  with  the  model,  as  well  as  the  structure  and  calculation 
of  the  probability. 


2.  Literature  Review 


There  exist  many,  varied  methods  of  calculating  the  probabilities  of  complex  systems  such 
as  the  one  seen  in  this  project.  Eor  the  purpose  of  this  research,  four  commonly  used  tools 
for  determining  probability  were  considered  and  are  discussed  below.  These  methods 
were  chosen  to  be  examined  based  on  literature  detailing  previous  study  done  in  similar 
fields.  Two  probabilistic  methods,  Bayesian  networks  and  logistic  regression,  and  two 
non-probabilistic  methods,  fuzzy  logic  and  neural  networks,  are  discussed  and  compared 
below  to  determine  which  gives  the  best  representation  of  the  system  involved  in  this 
project,  and  thus  the  best  outcome. 
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2.1  Bayesian  Networks 

Bayesian  techniques  are  based  on  Baye's  Law  which  relates  the  probability  of  event  a 
occurring  given  event  b  also  occurs  to  the  probability  that  event  b  occurs  given  a  has 
occurred,  all  in  the  context  of  background  information,  1.  That  is. 


pia\b,0 


p(f)|a,/)p(a,/) 

p(b,I) 


Bayesian  networks  are  a  way  of  graphically  representing  conditional  dependence  between 
random  variables,  and  are  analysed  using  probabilistic  methods;  specifically,  using 
Bayesian  probabilities.  The  variables  are  represented  as  vertices  in  a  directed  acyclic  graph 
(DAG),  with  the  edges  of  the  graph  designating  the  dependence  between  the  nodes.  As 
Bayesian  networks  are  a  probabilistic  technique  of  analysis,  they  will  generally  be  a  more 
accurate  measure  than  possibilistic  methods,  such  as  fuzzy  logic  discussed  below  [5]. 
Bayesian  inference  easily  accounts  for  subjectivity  and  conditionality  [6],  and  it  is  possible 
to,  without  losing  context,  deconstruct  the  system  in  order  to  analyse  certain  sections  [4]. 
However,  like  most  probabilistic  methods,  Bayesian  networks  are  computationally  heavy, 
although  they  tend  to  be  less  so  than  other  probabilistic  methods  [2]  [5].  When  used 
subjectively,  they  also  require  a  large  amount  of  cognitive  work. 


2.2  Logistic  Regression 

Logistic  regression  uses  Frequentist  probabilistic  methods  to  predict  the  probability  of  a 
dependent  variable  taking  a  certain  value  based  on  a  series  of  independent  variables.  Like 
Bayesian  networks,  it  can  be  assumed  that  this  method  has  greater  accuracy  than  others 
discussed  based  on  the  fact  it  is  probabilistic  [5].  In  addition,  it  is  capable  of  easily 
incorporating  interdependence  between  independent  variables.  However,  the  formulation 
of  a  logistic  regression  model  requires  a  certain  amount  of  holistic  sample  data,  which  is 
not  necessarily  readily  available.  Also  as  aforementioned,  logistic  regression  is  reasonably 
computationally  heavy  [2]  [5].  This  is  a  frequentist  method,  and  as  such  uses  the  frequency 
interpretation  of  probability.  Thus  logistic  regression  is  less  facilitating  when  accounting 
for  expert  opinion  and  subjectivity  than  methods  that  incorporate  conditional  probability 
and  Bayesian  probabilities  [6]. 


2.3  Fuzzy  Logic 

Fuzzy  logic  is  a  non-binary,  many-valued  logic  where  truth  values  can  fall  anywhere  in 
the  closed  interval  [0,1].  Its  form  of  reasoning  makes  it  simple  to  translate  linguistic 
concepts  into  quantitative  values,  and  it  is  a  good  method  for  sampling  data.  As  a  non- 
probabilistic  method,  fuzzy  logic  requires  less  computation  than  both  Bayesian  networks 
and  logistic  regression  [5],  and  is  also  efficient  in  approximating  functions  [3].  However,  it 
must  be  taken  into  account  that  as  a  possibilistic  method,  fuzzy  logic  is  a  much  less 
accurate  tool  of  measurement  than  probabilistic  methods,  such  as  the  two  discussed  above 
[5]. 
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2.4  Artificial  Neural  Networks 

An  artificial  neural  network  is  a  method  based  on  the  neurological  nervous  system  [7], 
involving  a  series  of  'neurons',  which  represent  variables,  connected  to  each  other.  Neural 
networks  differ  to  the  other  methods  considered  in  that  they  easily  incorporate  adaptive 
learning,  whereas  this  is  not  a  major  trait  of  the  others,  or  necessarily  possible  [7],  Because 
of  the  way  they  are  structured,  they  are  able  to  derive  complex  relationships  between 
variables  [8].  Neural  networks,  however,  use  unpredictable  procedures.  This  is  in  contrast 
to  the  other  methods  suggested,  which  are  entirely  predictable.  It  is  not  always  possible  to 
see  how  neural  networks  arrive  at  a  conclusion.  They  also  have  a  tendency  to  over  fit  the 
data,  and  are  a  computational  burden  [8]. 

2.5  Application 

This  particular  project  involves  the  prediction  of  success  of  a  given  complex  mission.  The 
data  involved  will  not  be  holistic  for  the  most  part  and  potentially  highly  subjective  in 
some  areas.  Because  of  these  reasons,  the  method  chosen  to  apply  to  this  project  is  a 
Bayesian  network.  Bayesian  inference  accounts  more  easily  for  subjectivity  than 
Frequentist  methods  such  as  logistic  regression.  In  addition,  the  ability  of  a  Bayesian 
network  to  break  a  system  down  and  evaluate  sections  without  losing  context  means  it  is 
appropriate  for  non-holistic  data,  such  as  what  may  be  provided  in  this  project.  While  the 
fuzzy  logic  method  is  computationally  the  most  efficient,  and  also  handles  inconsistent 
data  well,  it  tends  to  be  less  accurate  than  the  probabilistic  method.  Bayesian  networks 
have  been  known  to  be  used  in  similar  situations  in  the  past,  and  are  generally 
recommended  as  the  appropriate  statistical  method  for  decision  making  [2].  The  table 
below  shows  an  overview  of  the  abilities  of  each  of  the  methods  described  in  this  section. 

Table  1  Comparison  of  abilities  of  the  four  methods  described 


Bayesian 

networks 

Logistic 

regression 

Fuzzy  logic 

Artificial 

neural 

networks 

Probabilistic 

X 

X 

Accuracy 

X 

X 

X 

Ability  to  incorporate 
subjectivity 

X 

X 

X 

Ability  to  incorporate 
non-holistic  data 

X 

X 

X 

Computationally 

inexpensive 

X 

Predictable  procedures 

X 

X 

X 
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3.  Methodology 

3.1  Development  of  the  Bayesian  network  structure 

The  first  stage  in  developing  a  Bayesian  network  is  to  consider  the  events  relevant  to  the 
calculation  of  the  probability,  or  probabilities,  in  question.  By  definition,  a  Bayesian 
network  should  be  a  complete  representation  of  the  system  in  which  the  events  being 
measured  probabilistically  lie.  The  method  for  determining  what  is  relevant  depends  on 
the  situation,  and  in  particular  the  data  available.  If  a  large  amount  of  good  data  is 
available  for  a  particular  event,  it  may  be  possible  to  statistically  assess  the 
interdependency  between  events.  Bayesian  networks  are  able  to  account  easily  for  any 
subjectivity,  which  is  a  useful  trait  when  not  enough  data  is  available  to  assess 
quantitatively  the  likelihood  of  dependence. 

The  dependence  links  generated  from  the  first  step  are  represented  in  a  directed  acyclic 
graph  (DAG),  which  forms  the  Bayesian  network.  Vertices  in  the  said  graph  represent  the 
events  of  the  system,  and  are  otherwise  referred  to  as  nodes.  The  directed  edges 
connecting  the  nodes  are  dependent  relationships  between  events,  with  edges  running 
from  cause  to  effect  in  most  cases.  Events  which  cause  others  are  referred  to  as  parents, 
while  the  events  that  they  cause  are  known  as  children.  The  Bayesian  network  for  this 
project,  the  development  of  which  is  discussed  in  Section  4,  can  be  found  in  Appendix  A. 

3.2  Calculation  of  the  probability 

Not  all  of  the  events  in  this  network  will  necessarily  be  variable.  Some  are  fixed,  and  either 
determined  by  the  person  calculating  the  probability  in  question,  in  this  case  the 
probability  of  success,  or  by  the  environment  in  which  they  occur.  Let  F  =  {F^,  F2, ... ,  Em)  be 
a  set  of  fixed  events,  for  m  events.  F  Q  U  for  the  universal  set  U  of  all  events  in  the  system. 
It  should  be  noted  that  on  the  diagram  in  Appendix  A,  parents  of  fixed  nodes  have  not 
been  shown.  This  is  for  the  sole  purpose  of  simplifying  the  diagram,  although  it  should  be 
noted  that  a  complete  Bayesian  network  would  normally  display  these  links  as  well. 
However,  the  removal  of  these  links  has  no  effect  on  the  eventual  probability  calculated  as 
no  information  can  flow  through  fixed  nodes  from  their  parents.  As  these  events  are 
known  to  be  fixed,  their  parental  links  are  not  shown  in  the  diagram. 

Alternatively,  those  events  that  are  not  fixed  are  designated  variable  events.  Let  V  be  the 
set  of  n  —  1  variable  events,  such  that  V  =  F^  =  {Vi,  V2, ... ,  V  includes  events  such  as 
weather,  and  threats.  Variable  events  are  shown  on  the  diagram  in  Appendix  A  as  orange. 

Let  S  be  the  event  the  probability  of  which  is  being  calculated.  In  this  case,  S  is  the  event 
that  the  mission  succeeds. 

As  aforementioned,  this  project  aims  to  calculate  the  probability  of  mission  success,  S, 
given  certain  fixed  events,  fj.  This  is  expressed  as: 

P(S|f )  =  P(S|Fi  n  P2  n  ...  n  P^)  VP;  e  P,  i  =  1, ... ,  m. 
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The  law  of  total  probability  states  that: 

PiS\F)=Y,yPiS\V,F)P(V\F}. 

Let  Xi  be  any  variable  event  including S,  for  =  1,  It  is  now  possible  to  expand  the 
above  equation  in  the  following  manner: 

n 

P(S|f)  =  . . (1) 

V  i=l 

By  applying  the  chain  rule  for  Bayesian  networks  to  equation  (1)  can  be  simplified  so  it 
only  depends  on  the  parents  of  [9]. 


P(S|P)  =  in  Pi^i  IPi)  (2) 

V  i=l 

where  Pi  Q  U  is  the  set  of  all  parents  of  Aj.  Equation  (2)  is  the  general  form  of  the  equation 
used  to  calculate  the  probability  of  mission  success  in  this  project. 

Bayesian  networks  require  a  fair  amount  of  computation,  as  can  be  inferred  from  the 
general  equation.  To  calculate  the  probability  of  n  nodes,  with  each  node  i  having  a 
sample  space  of  Pj,  the  number  of  terms  t  in  the  summation  would  equal: 

n 

i=l 

If  every  node  could  only  take  on  two  possible  values,  the  number  of  iterations  any 
program  would  have  to  do  to  calculate  this  summation  would  be  2”,  and  it  would  be 
unrealistic  to  assume  that  each  node  will  have  a  sample  space  of  two.  While  Bayesian 
networks  do  reduce  the  amount  of  calculations  necessary  for  a  probabilistic  method,  there 
are  still  issues  with  the  computation  required  for  this  particular  method. 

Eor  each  variable  event  there  are  a  set  of  probability  distributions,  based  on  its  parents.  Eor 
example,  take  a  node  C  that  has  two  parents  A  and  B  as  follows: 


Figure  1  Diagram  of  a  simple  Bayesian  network  with  two  parents 
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Suppose  A  can  take  possible  values  {1,2},  B  can  take  {1,2,3}  and  C  {1,2}.  The  probability 
distributions  for  P{C  =  c\A  =  a, B  =  b)  for  a  E  A,b  E  B,c  E  C  are  represented  in  tables  a  n 
example  is  shown  in  Table  2. 


Table  2  Probability  distributions  ofC  given  A  and  B 


A=1 

B=1 

B=2 

B=3 

C=1 

0.1 

0.2 

0.3 

C=2 

0.9 

0.8 

0.7 

A=2 

B=1 

B=2 

B=3 

C=1 

0.4 

0.5 

0.6 

C=2 

0.6 

0.5 

0.4 

For  example,  from  this  it  can  be  determined  that  P(C  =  1|A  =  2,  B  =  1)  =  0.4.  There  must 
be  a  probability  distribution  for  C  given  all  combinations  of  a  and  b.  Note  that  to  be  a  valid 
probability  distribution,  the  individual  probabilities  must  satisfy: 

^ P(C  =  c\A  =  a, B  =  b)  =  1  \fa  E  A,b  E  B 
cec 

For  a  certain  aEA,bEB,cEC,an  element  in  the  matrix  will  represent  P(C  =  c|A  =  a,B  = 
b). 

However,  one  may  want  to  calculate  P{C  =  c)  regardless  of  the  values  of  A  and  B.  To  do 
so,  the  probability  distributions  of  A  and  B  are  required.  For  example: 


Table  3  Probability  distributions  of  A  and  B 


A=1 

0.1 

A=2 

0.9 

B=1 

0.6 

B=2 

0.3 

B=3 

0.1 

From  this  we  can  calculate: 

2  3 

P(c  =  1)  =  ^  ^  P(C  =  1|A  =  a,B  =  b)P(A  =  a)P(B  =  b)  =  0.42 

a=l  b=l 

This  example  demonstrates  the  basic  method  with  which  the  more  complex  network  for 
this  project  is  constructed  and  analysed. 
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4.  Application  of  method 

4.1  Construction  of  the  diagram 

For  the  commencement  of  this  project,  the  majority  of  dependence  links  were  determined 
with  expert  opinion.  The  activities  the  weapon  undergoes  throughout  the  mission  were 
derived  from  an  existing  model  of  the  system.  From  that,  possible  influences  on  the 
various  processes  were  considered.  A  chart  was  constructed  with  the  activities  that  the 
weapon  undergoes  and  their  known  influences  listed.  These  activities  and  influences  are 
represented  in  the  nodes  of  the  network.  Each  pairing  of  nodes  was  then  considered  in 
relation  to  each  other,  and  dependence  links  were  drawn  from  that  process.  Later  on,  new 
influences  may  be  introduced  and  the  probabilities  required  to  populate  the  network  will 
be  generated  more  rigorously.  However,  in  the  first  stage  of  developing  the  framework  for 
mission  success  measurement  it  is  not  necessary  to  do  so. 

In  this  situation,  the  Bayesian  network  is  structured  as  shown  in  Appendix  A.  This 
diagram  was  built  in  CORE®,  a  Model-Based  Systems  Engineering  tool,  and  it  displays  two 
categories  of  events:  activities  and  influences.  The  activities,  represented  by  operational 
activities  in  CORE®,  are  the  physical  processes  the  weapon  undertakes  on  the  mission.  The 
influences,  represented  by  operational  items  in  CORE®,  are  certain  environmental  effects 
and  characteristics  which  are  deemed  to  influence  the  mission.  In  general,  influences  will 
be  the  parents  of  activities,  but  as  shown  in  the  diagram  some  influences  affect  the 
probability  of  others. 

4.2  Calculation  of  the  probability 

A  Matlab®  tool  was  developed  to  calculate  the  probability  of  mission  success  using  a 
Bayesian  network.  The  inputs  of  this  function  are  the  fixed  events  in  F,  and  the  output  is 
the  probability  of  mission  success  (MS)  given  these  fixed  events,  or  P{MS\F).  For  this 
network,  there  are  eight  fixed  events,  which  are  to  be  decided  before  computing  the 
probability.  These  include  events  such  as  target  location  and  the  time  of  launch.  Further 
information  on  fixed  and  variable  events  can  be  found  in  Appendix  A.  By  calculating  the 
probability  of  success  for  various  times,  launch  points  and  other  controlled  events,  it  is 
possible  to  optimise  the  result  to  find  the  values  of  the  fixed  events  for  which  there  is  the 
greatest  likelihood  of  success. 

A  separate  function  was  created  to  demonstrate  the  calculation  of  the  collateral  damage 
estimate  (CDE).  This  was  done  similarly  to  the  method  used  to  calculate  mission  success. 
However,  the  CDE  demonstration  gives  a  vector  of  the  probability  distribution,  with  each 
element  corresponding  to  P(TAI  =  x\F)  for  each  x  G  TAI,  where  TAI  is  the  event  that  target 
area  interaction  is  at  a  certain  level  x.  The  sum  of  these  probabilities  should  equal  one.  As 
opposed  to  the  calculation  of  probability  of  mission  success,  where  collateral  damage  is 
assessed  in  a  binary  manner  where  the  damage  is  either  at  an  acceptable  level  or  not. 
However,  the  function  for  the  CDE  has  the  potential  to  convey  a  more  rigorous  and 
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detailed  assessment  of  the  level  of  collateral  damage  expected,  for  example  through 
detailed  modelling  and  simulation. 

4.3  Data  requirements 

What  is  necessary  in  terms  of  data  for  this  project  is  a  set  of  probability  distributions  for 
each  variable  event,  given  its  parents.  Whether  that  is  found  using  quantitative  methods, 
such  as  with  experimental  or  simulation  data,  or  whether  it  is  determined  via  subjective 
means  is  beyond  the  scope  of  this  part  of  the  project.  These  probability  distribution  tables 
are  input  into  Matlab®  in  the  form  of  matrices  of  dimension  p  +  1,  where  p  is  the  number 
of  parents  of  a  given  event.  Dummy  probability  distributions  were  constructed  for  the 
demonstration  of  this  method. 

In  the  demonstration  provided,  only  simple,  discrete  variables  were  used  for  the  example 
probability  distributions.  When  the  network  is  further  developed,  it  is  likely  the  system 
will  incorporate  continuous  probability  distributions.  Discretisation  of  these  variables  is  a 
method  of  working  with  the  distributions  on  a  practical  level,  but  both  fidelity  and 
computational  burden  need  to  be  considered  when  this  process  occurs. 

4.4  Reduction  of  unnecessary  computation 

As  mentioned  earlier,  a  major  practical  issue  with  Bayesian  networks  is  the  computation 
required  to  generate  probabilities,  especially  for  complex  networks.  In  order  to  minimise 
the  computational  time,  the  probability  of  all  activities  occurring  before  and  including  the 
initialisation  of  the  weapon  pre-separation  was  calculated  separately.  This  has  no  effect  on 
the  overall  probability  of  mission  success,  as  there  are  no  outside  influences  on  this  first 
stage  of  the  weapon  process  and  all  events  have  binary  outcomes  of  either  success  or 
failure.  Similarly,  mission  success  was  calculated  apart  from  the  majority  of  the  equation. 
This  also  has  no  effect  on  the  outcome  of  the  function.  To  reduce  the  number  of  iterations 
in  the  function,  a  method  was  devised  to  assess  whether  each  product  in  the  summation 
was  equal  to  zero  based  on  whether  any  of  the  individual  probabilities  calculated  were 
equal  to  zero.  These  products  are  unnecessary  for  the  summation,  and  thus  were  not 
computed. 

4.5  AgenaRisk® 

While  it  is  possible  to  use  Matlab®  or  other  similar  software  in  this  situation,  there  exist 
several  software  packages  specifically  designed  for  the  building  and  running  of  Bayesian 
networks.  For  this  project,  AgenaRisk®  was  used  to  reconstruct  the  network  developed. 
This  enabled  relatively  fast  calculation  of  the  probabilities  for  all  nodes  simultaneously.  It 
also  allowed  for  the  calculation  of  probability  propagation  in  the  network.  Running 
forward,  the  network  provides  a  prediction  of  events  of  child  nodes;  while  tracing  from 
child  nodes  to  parent  nodes  the  network  can  be  used  for  analysis  of  the  influence  of  parent 
nodes.  That  is,  if  a  probability  is  returned  that  is  considered  a  poor  result,  there  needs  to  be 
a  method  for  analysing  where  in  the  system  the  problem,  or  problems,  lie  and  thus  if  there 
is  potential  to  improve  the  mission  or  model  such  that  there  is  a  higher  likelihood  of 
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success.  Fixing  the  value  of  the  event  of  mission  success  to  an  observed  "false"  in 
AgenaRisk®  enables  the  user  to  observe  the  probability  of  all  events  occurring  before  that, 
given  that  the  mission  did  not  succeed.  In  doing  so,  it  is  possible  to  see  the  weak  points  in 
the  system. 

AgenaRisk®  accommodates  for  continuous  probability  distributions  by  discretising  the 
input  values;  the  size  of  the  intervals  is  determined  by  the  user.  Once  again,  the  amount  of 
computation  required  for  any  nodes  with  a  large  event  space  risks  becoming  an  issue,  but 
the  ease  with  which  the  number  of  intervals  can  be  changed  ameliorates  this  problem 
somewhat.  Figure  2  below  shows  a  simple  example  of  how  nodes  with  normal 
distributions  can  be  linked  in  AgenaRisk®. 


Figure  2  Diagram  of  a  simple  Bayesian  network  with  continuous  nodes 


5.  Conclusion 


This  report  details  the  development  of  the  framework  for  the  statistical  analysis  of  the 
probability  of  mission  success.  Firstly,  based  on  the  fact  it  is  a  probabilistic  modelling  tool 
easily  able  to  account  for  subjectivity  in  data,  a  Bayesian  network  was  chosen  over  other 
potentially  valid  but  less  well  suited  probabilistic  and  non-probabilistic  methods.  Given 
this,  the  structure  of  the  Bayesian  network  in  question  was  formulated  using  expert 
opinion  to  determine  any  dependent  relationships  and  necessary  influences  for  the  system. 
A  general  formula  for  the  calculation  of  the  probability  was  derived  and  then 
implemented  to  demonstrate  the  method  and  the  data  necessary  to  generate  the 
probability. 

5.1  Future  recommendations 

The  scope  of  this  report  covers  only  the  beginning  of  the  development  of  this  statistical 
measurement  tool.  The  next  stage  after  this  may  be  to  start  analysing  in  greater  detail  the 
data  necessary  to  calculate  mission  success.  In  particular,  the  level  of  precision  for  each 
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distribution  will  need  to  be  considered  in  regard  to  what  lies  in  the  event  space  for  each 
node,  or  more  specifically  the  size  of  the  event  space.  One  important  factor  to  keep  in 
consideration  while  doing  this  is  the  computational  burden.  On  that  topic,  the  further 
reduction  of  computation  time  is  also  something  that  should  be  considered  later  in  this 
project.  Doing  so  mathematically  requires  knowledge  of  the  specific  situation,  and 
methods  of  reduction  such  as  those  discussed  in  the  implementation  will  have  to  be 
determined  on  an  individual  basis.  It  would  also  be  prudent  to  start  collating  the  necessary 
data  for  measuring  the  likelihood  of  mission  success.  This  data  could  be  a  combination  of 
probability  distributions  calculated  through  Frequentist  means,  those  calculated  by  more 
subjective  methods,  or  both  depending  on  the  event  in  question. 

To  improve  on  the  work  currently  completed,  more  appropriate  software,  in  particular 
that  aimed  specifically  at  developing  and  calculating  Bayesian  networks  such  as 
AgenaRisk®,  should  be  investigated.  This  may  also  help  to  solve  the  problem  of 
computational  burden  as  the  software  will  generally  incorporate  algorithms  to  optimise 
the  calculations,  as  well  as  creating  a  single  location  for  all  information  regarding  the 
network.  Depending  on  the  software,  it  may  also  assist  in  measuring  posterior 
probabilities  as  mentioned  previously.  Software  such  as  this  may  be  used  to  validate  tools 
developed  separately  for  this  project,  or  to  validate  relevant  parts  of  those  tools.  Further 
exploration  of  the  capabilities  of  AgenaRisk®  is  also  recommended. 

No  method  has  been  developed  as  of  yet  to  measure  the  accuracy  of  the  results  obtained 
through  this  process.  Without  this,  the  tool  that  has  been  created  will  be  less  reliable.  A 
method  for  calculating  an  estimate  of  accuracy  would  have  to  be  implemented  to  be  able 
to  say  with  any  statistical  certainty  how  accurate  the  probability  presented  is,  and  this 
should  be  done  in  future  to  give  the  model  a  higher  level  of  credibility. 

While  continuing  this  project,  the  framework  of  the  Bayesian  network  should  be  put  under 
revision  at  relevant  stages.  This  includes  ensuring  all  dependencies  shown  are  relevant, 
potentially  by  more  formally  analysing  the  data  using  statistical  methods,  as  mentioned  in 
the  report.  It  should  also  be  remembered  that  not  all  the  influences  on  the  system  have 
necessarily  already  been  incorporated. 
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Appendix  A:  CORE®  Diagram 


This  CORE®  diagram  represents  the  Bayesian  network  developed  in  this  report.  Each  rectangular  node  is  an  activity  that  the  weapon  undergoes,  and  each  oval  node  is  an  influence  on  the  system. 

The  colours  of  the  nodes  represent  the  different  data  types,  as  follows: 

•  Orange:  variable. 

•  Blue:  fixed,  and  chosen  when  calculating  the  probability. 

•  Pink:  fixed,  but  uncontrolled. 
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